Semantic assessment of shopping behavior using trajectories, shopping related actions, and context information

The possibility of automatic understanding of customers’ shopping behavior and acting according to their needs is relevant in the marketing domain, attracting a lot of attention lately. In this work, we focus on the task of automatic assessment of customers’ shopping behavior, by proposing a multilevel framework. The framework is supported at low-level by different types of cameras, which are synchronized, facilitating efficient processing of information. A fish-eye camera is used for tracking, while a high-definition one serves for the action recognition task. The experiments are performed on both laboratory and real-life recordings in a supermarket. From the video recordings, we extract features related to the spatio-temporal behavior of trajectories, the dynamics and the time spent in each region of interest (ROI) in the shop and regarding the customer-products interaction patterns. Next ∗Corresponding Author. Phone: +31(0)15 2781537. Fax: +31(0)15 2787141. Email address: [email protected] (M.C. Popa) Preprint submitted to Pattern Recognition Letters December 24, 2011 we analyze the shopping sequences using a Hidden Markov Model (HMM). We conclude that it is possible to accurately classify trajectories (93%), discriminate between different shopping related actions (91.6%), and recognize shopping behavioral types by means of the proposed reasoning model in 95% of the cases.